Modern computing and display technologies have facilitated the development of virtual reality (“VR”), augmented reality (“AR”), and mixed reality (“MR”) systems. VR systems create a simulated environment for a user to experience. This can be done by presenting computer-generated imagery to the user through a head-mounted display. This imagery creates a sensory experience which immerses the user in the simulated environment. A VR scenario typically involves presentation of only computer-generated imagery rather than also including actual real-world imagery.
AR systems generally supplement a real-world environment with simulated elements. For example, AR systems may provide a user with a view of the surrounding real-world environment via a head-mounted display. However, computer-generated imagery can also be presented on the display to enhance the real-world environment. This computer-generated imagery can include elements which are contextually-related to the real-world environment. Such elements can include simulated text, images, objects, etc. MR systems also introduce simulated objects into a real-world environment, but these objects typically feature a greater degree of interactivity than in AR systems. The simulated elements can often times be interactive in real time.
FIG. 1 depicts an example AR/MR scene 1 where a user sees a real-world park setting 6 featuring people, trees, buildings in the background, and a concrete platform 20. In addition to these items, computer-generated imagery is also presented to the user. The computer-generated imagery can include, for example, a robot statue 10 standing upon the real-world platform 20, and a cartoon-like avatar character 2 flying by which seems to be a personification of a bumble bee, even though these elements 2, 10 are not actually present in the real-world environment.
Various optical systems generate images at various depths for displaying VR, AR, or MR scenarios. Some such optical systems are described in U.S. Utility patent application Ser. No. 14/555,585 filed on Nov. 27, 2014, the contents of which have been previously incorporated by reference herein. Other such optical systems for displaying MR experiences are described in U.S. Utility patent application Ser. No. 14/738,877, the contents of which are hereby expressly and fully incorporated by reference in their entirety, as though set forth in full.
Because the human visual perception system is complex, it is challenging to produce a VR/AR/MR technology that facilitates a comfortable, natural-feeling, rich presentation of virtual image elements amongst other virtual or real-world imagery elements. Improved techniques are needed for processing image data in such systems, including, for example, techniques for providing control data to control how the image data is displayed, techniques for correcting optical distortions in the image data, techniques for displaying and blending image data from many depth planes, and techniques for warping image data based on the head pose of a user. VR/AR/MR technology also has size and portability issues, battery life issues, system over-heating issues, and other system and optical challenges. Improved techniques are needed for addressing these issues, including, for example, overheat cause identification, time domain power management, discrete imaging mode, and eye/gaze tracking based rendering modification. The systems and methods described herein are configured to address these and other challenges.
What is needed is a technique or techniques to improve over legacy techniques and/or over other considered approaches. Some of the approaches described in this background section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued.